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This study proposed empirical indicators which can be validated and adopted in 
higher education institutions to evaluate quality of teaching and learning, and to 
serve as an evaluation criteria for human resource management and development 
of higher institutions in Thailand. The main purpose of this study was to develop 
empirical indicators of a theoretically-based multidimensional learners’ evaluation 
in Thai higher education adopting the National Institute of Development 
Administration as a case study. Research method of this study was developed 
based on Dubin’s Theory Building Model. Units, which are basic building blocks 
of newly developed concept, were developed qualitatively and quantitatively by 
triangulating data obtained from the systematic literature review, the qualitative 
data collection, and the Likert-scale survey. Patterson’s (in Holton and Lowe, 
2007) criterion was used to evaluate the conceptual framework. Questionnaire was 
used as a tool for data collection. Exploratory factor analysis and Cronbach’s alpha 
analysis were used to analyse the collected data in order to develop empirical 
indicators. 
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INTRODUCTION 

Higher education institutions have adopted several approaches and methods for quality 
improvement and evaluation as higher education institutions around the world have to 
respond to greater accountability and quality assurance (Hendry & Dean, 2002). In 
higher education setting, student evaluation or learners’ reaction has been studied since 
the 1920’s. The utility of learners’ reaction in formative and summative evaluation has 
been continuously confirmed by researchers (Wachtel, 1998). Student evaluation of 
teaching is an integral part of higher education practice (Shevlin et al., 2000). Despite 
the fact that student evaluation of teaching has been widely used, there have been 
theoretical issues that have not resolved yet, for example, dimensions to be covered in 
the evaluation (Shevlin et al., 2000). 
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According to the Office of Higher Education Commission of Thailand (OHEC), the 
evaluation of higher education quality is undertaken to help ensure the quality of the 
higher education institutions. The evaluation covers administrative systems and the 
mechanism of teaching and learning (Office of the Higher Education Commission, 
2007). Learners’ evaluation is counted as a part of indicators in the area of teaching and 
learning. Under the supervision of the OHEC, the National Institute of Development 
Administration (NIDA) continues to perform evaluation of educational quality using 
learners’ evaluation. 

The results of the evaluation are used for the purposes of identifying needed 
improvements in teaching and learning processes and those in human resource 
management actions, such as assessment of strengths and weaknesses of programs, 
identification of teaching technique and skill training for instructors, and development 
plans for instructors. Given its importance, the evaluation should be designed carefully 
in order to evaluate the quality of education properly and should yield valid results. 

LITERATURE REVIEW 

Sritanyarat (2014) developed a theory-based conceptual framework of multidimensional 
learners’ evaluation as an initial stage of developing teaching and learning quality 
evaluation tool for higher education. The study followed Dubin’s theory building 
method from the first to the fifth step, which will be discussed shortly. The indigenous 
conceptual framework consists of 25 units of evaluation. To utilize this theory-based 
conceptual framework, the development of empirical indicators is needed to improve the 
measurement of the values of each unit (Dubin, 1969; Lynham, 2002). 

This study takes on Sritanyarat’s (2014) initial conceptual framework to complete the 
sixth step of Dubin’s theory building method, namely empirical indicators development, 
with the purpose of providing higher education in Thailand, starting with NIDA, an 
evaluation tools for teaching and learning quality. 

Dubin's Theory-Building Method 

Dubin developed a widely-used theory building method (Holton & Lowe, 2007), and 
this method received attention in the field of human resource development (HRD) 
(Holton & Lowe, 2007). Dubin’s hypothetical-deductive theory building model is 
necessary for developing a valid and trustworthy applied theory (Lynham, 2002), or 
knowledge construction (Holton & Lowe, 2007; Lynham, 2002). 

Dubin proposed two different but related components of theory building: the theory 
development, and the research operation (Lynham, 2002). Completion of the first 
component results in a conceptual framework of the theory. Completion of the second 
component results in the development of valid and trustworthy applied theory (Holton & 
Lowe, 2007; Lynham, 2002). There are eight steps to complete these two components. 

The following are eight steps Dubin (1983) proposed: 
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i. Units are the basic building blocks, out of which the theory is built, or represent what 
the researcher is trying to make sense of (Dubin, 1983; Lynham, 2002). Units can also 
be referred to as variables (Dubin, 1983) 

ii. Law of interaction represents the nature of the relationships among the units 
(Lynham, 2002) 

iii. Boundaries determine and clarify the domains within the theory, where it is expected 
to hold up and apply (Lynham, 2002). 

iv. System states tell about conditions under which the theory is operative as a system 
(Dubin, 1983; Lynham, 2002). 

v. Propositions are specified to tell how a theory works in the real world (Lynham, 

2002 ). 

vi. Empirical indicators would be developed to secure the measurements of the values 
of the units (Dubin, 1969; Lynham, 2002). In other words, empirical indicators allow a 
theory to be tested. 

vii. Hypotheses are developed with the goal of establishing the link between the 
theoretical framework and the real world, by translating some of the propositions of the 
theoretical framework to testable statements (Lynham, 2002). 

viii. Testing is the last step which involves the testing of the theory through empirical 
research (Dubin, 1969). 

Research methods for this study, similar to those of Sritanyarat (2014), were developed 
following Dubin’s theory building model and Holton and Lowe’s (2007) concept in 
performing Dubin’s theory building model. 

The National Institute of Development Administration’s Teaching and Learning 
Evaluation 

Currently, the evaluation using the learners’ reaction-based evaluation is still a 
frequently-used method of learning evaluation (Yamnill, 2001) both in private and 
educational organizations, including NIDA. NIDA has established principles and 
methods for implementing the results of the teaching and learning evaluation of 
instructors to develop teaching and learning quality, and to develop instructors (National 
Institute of Development Administration. Personnel Department, 2012a). 

At present, NIDA performs a learning based evaluation of teaching and learning quality 
to serve the purpose of education quality assessment and internal human resource 
management. The evaluation uses a faculty evaluation form (FEF), which is completed 
by students at the end of each semester. The results of this evaluation are used for 
developing course administration quality, identifying strengths and weaknesses of the 
programs, identifying training needs for instructors, and so on. This evaluation affects 
instructors’ career as getting an evaluation rating lower than 3.51 can result in teaching 
suspension for instructors. Moreover, the evaluation results are included in promotion 
and salary increment consideration. 
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The current faculty evaluation form of NIDA covers the evaluation of instruction 
effectiveness, the faculty’s personal capability, and the use of teaching and learning 
media by the faculty. Considering all of the aspects of evaluation, it can be concluded 
that they cover only one dimension of learners’ evaluation, which is the instructor-based 
evaluation. The uni-dimensional evaluation approach is different from what was 
suggested by Morgan and Casper (2000) that a multi-dimensional approach to learners’ 
evaluation needs to be well designed in order to ensure the reliability. 

The following are details of NIDA’s faculty evaluation form, which is designed to be a 
self-report questionnaire for students to respond at the end of each course (NIDA, 
2012b). It is a three-page questionnaire. The purposes of the evaluation appear at the 
top of the first page of the evaluation sheet. Evaluators are asked to provide the 
following information: semester of evaluation, academic year of evaluation, section of 
evaluation, course code, course title, instructor’s name, school, level of degree, and 
program type (NIDA, 2012b). Students are not asked to provide information that could 
be used to identify them personally. 

The form consists of 25 Likert-type questions, each of which includes a not applicable 
option. These questions cover three aspects, and there are nine open-ended questions 
covering the three aspects. 

The first aspect covered in the form concerns the effectiveness of the teaching and 
learning. There are seven Likert-type questions related to the first aspect. The second 
aspect covered in the form involves the faculty member’s personal capability. There are 
seven Likert-type questions related to the second aspect. The third aspect covered in the 
form involves the teaching and learning media aspect. There are three Likert-type 
questions that measure the use of learning media as a whole. There are another eight 
items that measure the use of E-learning media (National Institute of Development 
Administration. Personnel Department, 2012a). As per the open-ended section of the 
form, there is one question that asked about the effectiveness of E-learning. This 
question, along with other open-ended questions, appeared right after the Likert-scale 
questions. Other open-ended questions asked for opinions about problems and 
obstacles, as well as suggestions about E-learning. 

Students are required to complete the faculty evaluation form for every course they take. 
Failing to do so results in 30-day withholding of grades announcement for that particular 
semester. 

METHOD 

According to Sritanyarat (2014), the first implication for future research was to develop 
and refine empirical indicators of their newly developed theory. This study followed the 
six steps of Dubin’s theory building, as proposed by Sritanyarat (2013). More 
specifically, the six steps of Dubin’s theory building model were taken by adapting 
Holton and Lowe’s (2007) concept in performing Dubin’s theory building model from 
step one to step five Sritanyarat’s (2013) research process was presented in the middle 
column of the following figure. 
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This study took on Sritanyarat’s (2014) conceptual framework of multidimensional 
learners’ evaluation. Specifically, empirical indicators were developed by the 
systematic approach suggested by Hinkin (2005). 

According to Sritanyarat (2014), there are 25 units included in the framework. Units are 
put together in groups of learners-related units, instructors-related units, course-related 
units, and supporting element-related units. The following are the 25 units. 

xiii. Helpfulness 

xiv. Expressiveness and rapport 

xv. Teaching methods 

xvi. Feedback 

xvii. Instruction qualification 
xviii. Instrument clarity 

xix. Instrument organization 

xx. Interest 

xxi. Engagement with learning and active 
participation (motivated) 

xxii. Positive attitude and good feeling of 
learning 

xxiii. Environment of teaching and learning 

xxiv. Resource and facility availability 

xxv. Learners’ perception of learning 

Taking all 25 units, the following was the procedure used for items generation. The 
following are steps in research method. 

Developing empirical indicators 

Item generation, wording, and scaling 

To generate items to represent empirical indicators, every unit and its elaboration 
provided by Sritanyarat (2014) were used to acquire theoretical definitions. Literature 
reviews were also conducted to develop theoretical definitions for every conceptual 
dimension and to generate items out of those definitions. There were 46 items initially 
developed. 

Items needed to be initially developed in English as it represented the language for the 
theoretical conceptual framework and the literature that had been reviewed. Back 
translation was performed to translate into the Thai language, which was the first 
language for NIDA students who were the expected survey participants. The back 
translated version was put together with the English version so that evaluators could 
choose to perform their evaluation in either languages at will. Likert-type items with a 


v. 

vi. 


Course characteristic and qualification 
in concentration on the fundamental 
concept 

Utility judgment about the course 
Instructors’ knowledge and expertise 
Course content 
Course evaluation 
Course material 


vii. Workload and requirement of the 
course 


x. 

xi. 

xii. 


Overall characteristics 
qualification of instructors 

Easiness 

Respect for learners 

Enthusiasm 

Accessibility 


and 
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five-point scale were selected, since they can increase coefficient alpha reliability 
(Hinkin, 2005). The term used for the does not apply option was Unable to evaluate. 

Content validity assessment 

Content validity was assessed by the concept of indexes of item-objective congruence 
(IOC) to evaluate content validity as mentioned by Turner and Carlson (2003). The 
recent Standards for Educational and Psychological Testing (2014) indicated that, 
although validity is considered a unitary concept, there are different forms of evidence. 
One type of evidence is evidence based on the content. In this case, experts in the field 
examined parts of the instrument in relationship to the concept being measured. 

After analysis of the IOC results, changes were made according to the suggestions from 
experts, who were stakeholders of teaching and learning in NIDA including instructors, 
support officers, and students. Items were excluded, collapsed, and modified, and 
changes were made in verb selection. After this stage of content validity assessment and 
modification, there were 39 items included in the questionnaire for students to complete. 

Testing the empirical indicators 

Target population included all 7,111 students of NIDA (National Institute of 
Development Administration, 2011b). According to Krejcie and Morgan’s (1970) 
suggestion, for a population of 8,000, 367 respondents are needed. Questionnaires were 
distributed to students at the end of the second semester of the academic year 2012. 
After data collection was performed, the item-to-response ratio was 1:25.2778. The 
total number of respondents was 910. 

The following are reports on respondents' demographic data including type of program 
in which respondents were enrolled, type of courses respondents were evaluating, and 
grade point average of respondents. The demographic data shows that respondents from 
regular programs were lower in percentage (30.66%) compared to those from executive 
programs (66.04%), which aligns with NIDA's percentage of students from regular 
programs and executive programs. Type of courses was grouped into three categories: 
core course (44.62%), major course (33.41%), and elective course (21.87%). The ratio 
shows a somewhat similar percentage of respondents who were taking core courses and 
major courses, while those who were taking elective courses showed a somewhat lower 
number of responses. Respondents' learning performance in the form of grade point 
average was grouped according to NIDA's grading system. NIDA’s grading system is as 
follow. The maximum grade point is 4.00, which equals to the grade mark of A. The 
grade mark of A- refers to 3.70. B+ refers to 3.30. B refers to 3.00. It was found that 
most of the respondents had a GPA of 3.30-3.70 (40.44%). Preliminary factor analysis 

Exploratory factor analysis (EFA) with a varimax rotation was used to explore what 
factors were of the theoretical conceptual framework of multidimensional learners' 
evaluation using the data from NIDA’s students. [A varimax rotation, as one of the 
most common rotations, results in orthogonal factors that maximize the variance of 
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those factors.] The following shows findings about factors that emerged when using the 
multidimensional learners' evaluation in teaching and learning evaluation at NIDA. This 
study adopted a minimum loading of .40, which was considered .10 higher than the 
minimal level (Hair et al., 1998). 

Internal consistency assessment 

Internal consistency reliability was evaluated and reported by using Cronbach’s alpha 
(Price & Mueller, 1986). This step was performed to allow the tailoring of scales to 
have the same number of items and still retain adequate domain sampling by carefully 
examining each item (Hinkin, 2005). 

FINDINGS 

After performing the principles and practices in scale development suggested by Hinkin 
(2005) in developing the empirical indicators, slight changes were made in the items that 
were retained. One item was not retained as it yielded the factor loading of lower than 
.40. Two items were excluded, because they were developed to be course related, but 
the EFA put them in the factor of instructor’s teaching skills. However, the five factors 
from the initial conceptual framework retained: instructor’s behaviors and interpersonal 
skills, learners, instructor’s teaching skills, course, and supportive elements. The 
following table reports the retained factors, units, and the number of items. 

Table 1: Retained factors, units (of the 25 units of evaluation), and item components 


Retained Factors 

Number of Units 

Number of Items 

Instructor’s Behaviors and Interpersonal Skills 

8 

11 

Learners 

5 

10 

Instructor’s Teaching Skills 

5 

7 

Course 

4 

5 

Supportive Elements 

1 

3 

Total 

23 

36 


Within the factor of Instructor’s Behaviors and Interpersonal Skills the following units 
appear: knowledge and expertise, overall characteristics and qualifications of work 
ethics and punctuality, easiness, respect for learners, enthusiasm, accessibility, 
helpfulness, and expressiveness and rapport. An example of one of the items is the 
instructor shows open-mindedness to different opinions, and respects different opinions. 
Within the factor of Learners are the unites: interest, engagement in learning and active 
participation (motivated), positive attitude and good feeling about learning, environment 
of teaching and learning, and learners’ perception of learning. An example of one of the 
items is in classes, your classmates are engaged with learning, active participating and 
interacting with instructor. Within the factor of Instructor’s Teaching Skills there are 
five units: teaching methods, feedback. Instruction qualification, instrument clarity, and 
instrument organization. An example of an item is the instructor provides feedback to 
learners. The factor of Course includes these units: course characteristics and 
qualification in concentration on the fundamental concept, utility judgment about the 
course, and course content. An example of an item is this course can contribute to the 
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preparation for learners' future work. The final factor of Supportive Elements includes 
one unit: resource and facility availability. An example item is support material and 
library resources that met with needs for the subject matter, such as internet system, 
database, computer software, are available and appropriate. 

Reliability coefficients were determined for each factor, as recommended by Hinkin 
(2005). According to Tavakol and Dennick (2011), the minimum acceptable numeric 
values of Cronbach’s alpha ranged from .70 to .95. A Cronbach's alpha of .961 was 
found. After performing the reliability coefficient test, one item was excluded, as the 
correlated item-total correlation of .240 was found. Overall alpha results were higher 
when the item was deleted. After excluding the item the Cronbach's alpha of .963 was 
revealed, which could be described as an acceptable internal consistency. The 
followings are reports on reliability coefficient test obtained by using Cronbach's alpha 
of each factor considering only items belonging to the particular factors. 


Table 2: Reliability coefficient for every individual factor 


Factor 

Number of 

Correlated Item- 

Alpha if 

Correlation 

Items 

Total Correlation 

Item Deleted 

Matrix Range 

Instructor’s Behaviors 

and Interpersonal Skills 

11 

.936 

.926-.939 

,2725-.7743 

Learners 

10 

.922 

.913-.924 

.340-.762 

Instructor’s Teaching 

Skills 

7 

.937 

.925-.932 

.629-.762 

Course 

5 

.866 

.829-.856 

.441-.693 

Supportive Elements 

2 

.654 

- 

.486 

Total 

35 

- 

- 

- 


After completing the EFA and internal consistency assessment, five factors, 23 units 
with 35 items were retained. 

DISCUSSION 

Multidimensionality 

This study proposes the use of multidimensional learners' evaluation tool for higher 
education in Thailand. The proposed concept aligns with what was suggested by 
Morgan and Casper (2000): that a multi-dimensional approach of reaction evaluation 
needed to be well designed to ensure the reliability of learners’ evaluation. The final 
conceptual framework includes five factors, namely course, supportive element, 
instructor’s teaching skills, instructor’s behaviors and interpersonal skills, and learners. 
Those five factors could be regarded as four dimensions of teaching and learning 
evaluation, namely, course-related, instructor-related, learners-related, and supportive 
element-related. There are two factors that fall in the dimension of instructor-related: 
instructor’s teaching skills, and instructor’s behaviors and interpersonal skills. 

Discussion pertaining to the existing literature and NIDA Faculty Evaluation Form 
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This study supports Sritanyarat’s (2014) proposition that teaching and learning 
evaluation is multidimensional by separating issues of course-related, learner-related, 
and supporting element-related from those of instructor-related issues. According to the 
results, the focus of teaching and learning evaluation is not only on instructors, but also 
on learners, as well as course and the supportive environment since courses and learning 
environments are not designed solely by instructors. The units, which were the basic 
building blocks represent what the researcher tried to make sense of (Dubin, 1983; 
Lynham, 2002), obtained from the systematic literature review, items of NIDA’s Faculty 
Evaluation Form (FEF), and units proposed in the results of this study are compared and 
discussed. 

Discussion of the confirmed units 

The NIDA FEF includes six units, all of which are instructor-related: instructors' 
qualification of instruction, instrument organization of instructors, instrument clarity of 
instructors, overall instructors' characteristics and qualification, accessibility of 
instructors, and teaching methods of instructors. The present study confirms these units 
within the factors identified in the factor analysis: instructor’s behaviors and 
interpersonal skills, and instructor’s teaching skills. All six units are what instructors 
can perform to ensure the quality of teaching and learning. 

In addition to NIDA’S FEF, the OHEC in Thailand has identified indicators of higher 
education quality. One indicator focuses on system and mechanism of teaching and 
learning administration (National Institute of Development Administration, 2011a) and 
this suggests that every program should have a learning system and mechanism that 
encourage learners-centered concept. Moreover, every program must provide learning 
plan for each course before the semester begins. Each course of the program should be 
designed to integrate body of knowledge from the field of practice. Fearning should be 
both classroom-based and non-classroom approach. Research must be incorporated in 
every course. Learners’ evaluation results must be reported. The minimum passing 
score is 3.51 out of 5.00. Moreover, every course must be developed according to the 
evaluation results (National Institute of Development Administration, 2011a). 

NIDA’s current teaching and learning evaluation incorporates the above units and uses 
them as points for development. Thus, it is reasonable and important for the results of 
the present study to agree with both the NIDA FEF and the OHEC’s number 2.6 quality 
indicators. 

Discussion of factors and units emerging from the study 

Three factors emerged that go beyond NIDAs FEF, specifically course, learners, and 
supportive environment. Within each of the factors appear specific units. Three course- 
related units emerged: overall course quality and characteristics, course content, and 
utility judgment. There are five learner-related units: interest in learning of learners, 
environment for teaching and learning, learners' positive attitude and affect or 
satisfaction, learners' engagement with learning and active participation, as well as 
learners’ perceptions of learning. Finally, two supportive units emerged: resources 
availability, and technology availability. 
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The course-related, learner-related, and supportive element-related factors and units 
appeared in this study but are not included in NIDA’s FEF. Such results probably 
appeared because NIDA’s FEF puts its focus on only e-learning as part of teaching 
methods unit. 

As part of the instructor behaviors and interpersonal skill factor units there are seven 
instructor-related units. It should be noted that NIDA’s current evaluation only focuses 
on instructor behaviors and does not consider personal or personality characteristics. 
Nevertheless, some of the units listed below that emerged from the current study suggest 
that both behaviors and characteristics are of importance. 

i. Helpfulness of instructors emerged from the results of this study. It could be 
discussed that this emerged as Thai learners are teacher-centered (Johansen and 
Gopalakrishna, 2006; Komin, 1990). It is likely that Thai students would appreciate 
instructors’ being helpful. 

ii. Instructor’s expressiveness and rapport was revealed by this study. It refers to 
instructors’ consideration for learners, expressiveness and interaction with learners, and 
ability to develop rapport with learners. Thai students’ value of an appreciative 
orientation, as suggested by Komin (1990), would suggest that an instructor being 
expressive and cordial is of concern. 

iii. Enthusiasm of instructors was also revealed by this study. It refers to the 
instructor being genuinely interested in teaching (Zhao & Gallant, 2012). This does not 
solely relate to the instructor’s action, but also refers to some personality characteristics 
of the instructor. 

iv. Feedback providing is a behavioral unit revealed by this study. The current 
NIDA’s FEF includes an item concerning the instructor being able to advise students via 
appropriate channels. This item could be considered as partially related to the element 
of feedback providing. 

v. Respect for learners by instructors was found from the results but is not included 
in NIDA’s current evaluation form. It is possible that this emerged due to the data 
collection from students and officers from graduate schools. NIDA’s current evaluation 
was designed without input from students and officers from graduate schools, who work 
closely to students. This subject might come to an interest because of Thai’s value 
system. According to Komin (1990), Thai people are ego oriented and are not able to 
tolerate ego violations or offenses. Even when they are taking the role of student; they 
prefer to earn respect from instructors, especially, when they consider themselves as 
adults and in higher education system. 

vi. Easiness of instructors emerged from the results of this study. Easiness which is 
emerging here does not refer to instructors as easy graders. It refers to whether 
instructors are easy to talk to, whether they are kind and supportive, as well as whether 
they provide a positive relationship with learners. It is highly possible that it emerged 
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due to the input of students and graduate school officers. It may be of concern because 
of Thai’s value system as well. Thai’s prefer to maintain smooth relationships (Komin, 
1990). Consequently, instructors who are easy to approach would be more preferable in 
order to have a qualified learning experience for learners. 

vii. The instructor’s knowledge and intellect are characteristic units proposed by this 
study. Some may question whether learners are competent to evaluate their instructor’s 
knowledge. However, it is obviously a concern and expectation of Thai’s value system. 
Instructors are valued as superiors, as Thai culture values good grades and high levels of 
education, as well as a focus on labels in social relations (Komin, 1990). When 
instructors are considered superior, expectations are placed on them to exhibit their 
knowledge. 

Discussion of the units found in NIDA’s FEF which is not con firmed by the results of 
this study 

There is one unit which was found only in FEF which was not supported by the results 
of this study: the use of proper research results or academic services in the course 
content. This unique item could be derived from the objective of NIDA as being a 
research university; therefore, it is likely that NIDA’s FEF includes this unit. It may be 
that learners or students do not view NIDA as being a research university. Research 
may not be a major concern of students, as most of them are in the field of practice. 

Additionally, there are four units which cover many items of FEF, namely instrument 
organization of instructors, instrument clarity of instructors, overall instructors' 
characteristics and qualification of work ethics, open-mindedness, and punctuality, and 
teaching methods of instructors. They are basically evaluated altogether. However, this 
study suggests that they should be evaluated separately. 

LIMITATIONS 

This study takes on the conceptual framework proposed by Sritanyarat (2014) to 
develop empirical indicators to complete the research process suggested by Sritanyarat 
(2013). It is possible that there are more units and laws of interaction that have not 
been uncovered yet. 

Three groups of stakeholders in NIDA - students, instructors, and officers or staff — 
were involved in this study. It is possible that different groups of stakeholders, such as 
employers, higher levels of administration, or government officials, could provide 
different aspects on higher education evaluation (Green, 1994; Tam, 2001). To delimit, 
this study can be replicated and extended by inclusion of more groups of stakeholders in 
the qualitative data collection. Employers could be involved as users of graduates from 
NIDA. Top management of NIDA and alumni could be included as another group of 
stakeholders. 

The data collection about teaching and learning evaluation is considered sensitive. Data 
regarding the school names, program names, instructors’ names, and other information 
that could be used to identify the instructors of the courses were not allowed to be 
collected. Therefore, analysis using the previously-mentioned data, such as comparative 
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analysis among the different schools, could not be performed. Moreover, different 
schools had different preferences in assigning groups of respondents and the researcher 
needed to comply with the schools’ preference in collecting the data. Future studies 
may be undertaken using similar groups but from different schools or higher education 
institutions. 

RECOMMENDATIONS FOR FUTURE RESEARCH 

Firstly, the development and then testing of the hypotheses are needed to complete 
Dubin’s model. The step of developing a hypothesis represents an establishment of the 
link between the theoretical framework and the real world, by translating some of the 
propositions of the theoretical framework to testable hypotheses (Lynham, 2002). One 
possible hypothesis might be that the instructor characteristics have a direct effect on all 
other dimensions. 

Secondly, empirical research studies are needed to validate the conceptual framework in 
the real work context. This step involves the testing of the theory through empirical 
research. The results of this study can be adopted as variables in some of the future 
empirical studies. 

Thirdly, replication of this study could be performed in other higher education 
institutions in Thailand for the purpose of further generalization. There is a possibility 
that different contexts of higher education, even those in Thailand, hold different 
characteristics from NIDA. Furthermore, beyond Thailand, replication of the study 
could be undertaken in higher education institutions in other countries and cultures. 

In order to complete Dubin’s theory for making method, further research needs to be 
done. The development and testing of the hypotheses would complete the theoretical 
conceptual framework and would make a greater contribution to both academia and 
practice. 

IMPLICATIONS FOR PRACTICE 

This study proposed and developed an integrated framework and questionnaire which 
can be considered as a tool for multidimensional learners' evaluation to assess the 
quality of teaching and learning in higher education. The emerging factors and units of 
the framework can be taken as issues of concern in evaluating quality of teaching and 
learning in higher education in Thailand, as well as other more general settings. 

The results of this study can be utilized in human resource management mechanism of 
NIDA, as well as other higher education institutions in Thailand. Learners’ evaluation is 
undertaken for the purpose of human resource management of NIDA faculty and 
instructors (National Institute of Development Administration. Personnel Department, 
2012a). The questionnaire obtained from this study can be adopted for use in teaching 
and learning evaluation as it was developed with indigenous data and context. For other 
higher education institutions in Thailand, it is important to verify the questionnaire 
within the specific context with specific stakeholders or an expert panel before 
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performing the data collection. However, to use the questionnaire, it is recommended to 
consider the length of the questionnaire, since it can affect its’ reliability and validity. It 
is recommended that mechanism and system of evaluation should be carefully designed 
to promote interest of learners in completing the questionnaire, or to compel learners to 
complete the questionnaire. 
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Turkish Abstract 

Thai Yiiksekogretimi’nde Ampirik Degerlendirme Gostergeleri: Teori-Temelli (,'ok Bovutlu 
Ogrenci Degerlendirmesi 

Bu 5 ah§ma Tayland’da yuksekogretim kurumlarimn geli§imi ve insan kaynaklarinin yonetimi iqin 
degerlendirme kriteri olarak kullanilmak ve yiiksekogretimde ogretmen ve ogrenmenin kalitesini 
degerlendinnek i 9 in ge 9 erlenebilecek ve uyarlanabilecek ampirik gostergeler sunmayi 
ama 9 lamaktadir. Bu 9 ah:jmanm ana amaci Ulusal Geli§im Yonetimi Kurumunu bir ornek olay 
olarak alarak Thai yuksekogretiminde teori-temelli 90 k boyutlu ogrenci degerlendirmesinin 
ampirik gostergelerini geli§tinnektir. Bu 9 ali§manm yontemi Dubin Teori Geli§tinne Modeline 
dayamlarak geli§tirihni§tir. Yeni geli§tirilen kavramin temel yapi ta§lari olan liniteler sistematik 
literatiir taramasindan, nicel veri toplamadan ve Likert ol 9 ekli anketlerden elde edilen veri 
ii 9 lemesi yoluyla nitel ve nicel olarak geli§tirilmi§tir. Kavramsal 9 er 9 eveyi degerlendinnek i 9 in 
Patterson’un (Holton and Lowe, 2007 i 9 inde) ol 9 Utu kullamlmi§tir. Veri toplama araci olarak 
anket kullamlmi$tir. Ampirik gostergeleri geli§tirmek i 9 in toplanan veriyi analiz etmede 
a 9 imlayici faktor analizi ve Cronbach Alfa analizi kullamlmi§tir. 

Anahtar Kelimeler: Ogrencinin degerlendirmesi, yuksekogretim, Tayland, Dubin, teori geli§tinne 

French Abstract 

Indicateurs devaluation Empiriques dans Enseignement Superieur Tha'hl'Evaluation 
d’Apprenants Multidimensionnels A Base de Theorie 

Cette etude a propose les indicateurs empiriques qui peuvent etre valides et adoptes dans des 
institutions d'enseignement superieur pour evaluer la qualite d'enseignement et l'apprentissage et 
servir de criteres devaluation pour la gestion des ressources humaines et le developpement 
d'institutions plus hautes en Thailande. Le but principal de cette etude etait de developper les 
indicateurs empiriques d'une evaluation des apprenants multidimensionnels theoriquement bases 
dans l'enseignement superieur tha'i adoptant l'Institut national d'Administration de Developpement 
comme une etude de cas. La methode de recherche de cette etude a ete developpee basee sur la 
Theorie de Dubin Construisant le Modele. Les unites, qui sont les composantes de base de 
concept nouvellement developpe, ont ete developpees qualitativement et quantitativement en 
triangulant des donnees de l'examen systematique de litterature, la collecte de donnees qualitative 
et l'enquete de Likert-echelle. Patterson (dans Holton et Lowe, 2007) le critere a ete utilise pour 
evaluer le cadre conceptuel. Le questionnaire a ete utilise comme un outil pour la collecte de 
donnees. L'analyse de facteur exploratoire et l'analyse alpha de Cronbach ont ete utilisees pour 
analyser des donnees rassemblees pour developper des indicateurs empiriques. 

Mots Cles: revaluation d'apprenants, enseignement superieur, la Thailande, Dubin, construction 
de theorie 
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German Abstract 

Empirische Bewertungsindikatoren im Thailandisch Hohere Bildung: Theoriebasierte 
Bewertung der Multidimensionalen Lerners 

Diese Studie vorgeschlagene empirische Indikatoren, die in Hochschuleinrichtungen validiert und 
angenommen werden kann, Qualitat der Lehre und des Lernens zu bewerten und als 
Bewertungskriterien fur Personalmanagement und Entwicklung der hoheren Institutionen in 
Thailand zu dienen. Der Hauptzweck dieser Studie war es, empirische Indikatoren fur die 
Bewertung eines theoretisch-basierten multidimensionalen Lernenden in Thai hoheren Bildung zu 
entwickeln. Wir nahm die National Institute of Development Administration als Fallstudie. 
Forschungsmethode dieser Studie wurde auf Basis von Dubin Theorie Gebaudemodell entwickelt. 
Einheiten, die neu entwickelte Konzept Grundbausteine sind, wurden qualitativ und quantitativ 
entwickelt von Daten aus der systematischen Literaturrecherche, die qualitative Erhebung von 
Daten und der Likert angelegte Umfrage Triangulation. Patterson (in Holton und Lowe, 2007) 
Kriterium wurde verwendet, um den konzeptionellen Rahmen zu bewerten. Fragebogen wurde als 
ein Werkzeug zur Datenerfassung benutzt. Explorative Faktorenanalyse und Cronbachs Alpha- 
Analyse wurden eingesetzt, um die gesammelten Daten analysieren, um empirische Indikatoren 
zu entwickeln. 

Schliisselworter: bewertung der lernenden. hochschulbildung, Thailand, dubin, theoriebildung 

Malaysian Abstract 

Penilaian Empirikal Petunjuk Pengajian Tinggi di Thailand: Teori Berasaskan Penilaian 
Multidimensional Pelajar 

Kajian ini mencadangkan petunjuk empirikal yang boleh disahkan dan diterima pakai di institusi 
pengajian tinggi untuk menilai kualiti pengajaran dan pembelajaran, dan untuk berkhidmat 
sebagai kriteria penilaian bagi pengurusan sumber manusia dan pembangunan institusi pengajian 
tinggi di Thai. Tujuan utama kajian ini adalah untuk membangunkan petunjuk empirikal 
berasaskan teori pelbagai dimensi pelajar dalam pendidikan tinggi Thai dengan menggunakan 
Institut Tadbiran Pembangunan sebagai kajian kes. Kaedah penyelidikan kajian ini teiah 
dibangunkan berdasarkan Dubin’s Theory Building Model. Unit, merupakan blok binaan asas 
konsep baru yang dibangunkan, teiah dibangunkan secara kualitatif dan kuantitatif melalui 
triangulasi data daripada kajian literatur yang bersistematik, pengumpulan data secara kualitatif 
dan tinjauan menggunakan skala Likert. Patterson (dalam Holton & Lowe 2007) kriteria 
digunakan untuk menilai rangka kerja konseptual. Soal selidik digunakan sebagai alat untuk 
pengumpulan data. Analisis faktor penerokaan dan analisis alpha Cronbach teiah digunakan 
untuk menganalisis data yang dikumpulkan untuk membangunkan petunjuk empirikal. 

Kata Kunci: penilaian pelajar, pendidikan tinggi. Thailand, Dubin, theory building 
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